Author
|
Topic: Are time bars necessary
|
J L Ogilvie Moderator
|
posted 06-02-2005 12:25 PM
Are time bars really necessary on pre-employment tests using a comparison question format?There is no specific situation to separate from. If they are hiding anything it could be from any point in their life so couldn't you leave off the time bars? During discussions around here it seems as though people are identifying comparisons in some instances by the time bar then employing counter measures. Why not leave it off? You might not want to ask them if they ever lied to someone in authority because they may have lied to the background investigator but that leaves alot of other questions you could use without the time bars. What do you think? Jack ------------------
IP: Logged |
J.B. McCloughan Member
|
posted 06-02-2005 02:41 PM
It is my understanding from training and in reading research that time bars are not necessary. In fact one research study done on the use of exclusive (time bar) and nonexclusive (no time bar) comparisons found that the nonexclusive had significantly better results in correct decisions (+32%), false positives (-15%), and inconclusive (-18%) but had a slight increase in false negatives (+1%). This study is on criminal specific single issue testing though.[This message has been edited by J.B. McCloughan (edited 06-02-2005).] IP: Logged |
Barry C Member
|
posted 06-03-2005 11:57 AM
First, I don't think they are necessary (based on the research out there), but it's one of those things that many like to see because, intuitively, they should help.With regard to pre-employment exams, I think it depends on where you are in the process as to whether you can use them effectively or not (assuming for the moment that they actually do what is intended). During a screening exam, I think you're right (unless your RQs are as an adult and the CQs are before age 18 or something like that). But, if you narrow down one of those issues and plug it into a single-issue test format, you could use something like, "Excluding this issue...? I use disguised comparisons (I think Jim Wygant called them "future" CQs) that don't require bars such as "Are you the type of person who... would lie to a person who trusts you ...is completely trustworthy ... would lie to cover up a mistake ...etc." IP: Logged |
Capstun Member
|
posted 06-03-2005 12:37 PM
I agree and have dropped the time bars on screening test comparisons. I put them on the irrelevants, which has caused a few people to employ contermeasures on these questions IP: Logged |
Barry C Member
|
posted 06-03-2005 12:44 PM
I do the same thing, but I haven't got any bites yet. I even introduce them as "controls."IP: Logged |
sackett Moderator
|
posted 06-03-2005 01:11 PM
Jack,I think J.B.M. is correct, there is some literature/research out there addressing the need (or not) of time bars (or other separation of CQ from RQ's). In regards to validity, it is the standard set by most schools/training and how most reliable research was conducted. But, you might want to get the opinion of a real academic. LOL Jim IP: Logged |
Barry C Member
|
posted 06-03-2005 02:30 PM
The Reid test is considered "valid" and it doesn't use time bars.Since we're talking screening exams, validity isn't much of an issue - utility is, and it would seem adding time bars in a test that covers one's lifetime doesn't make any sense. IP: Logged |
J L Ogilvie Moderator
|
posted 06-03-2005 04:59 PM
Good answers one and all. I feel very much the same way. This was a discussion we were having the other day so I thought I would run it up the flag pole and see who saluted.I would like someone from DODpi to weigh in on this issue. How about it? By the way, we all here introduce the irrelevants as "controls" or "comparisons" and several times have had counter measures tried on those questions. We love it. Jack ------------------
IP: Logged |
Barry C Member
|
posted 06-03-2005 07:17 PM
It was Don Krapohl of DoDPI who told me about the "Are you the type..." CQs above, and I understand they use them in the LEPET taught by DoDPI (and published on George's website).IP: Logged |
Ted Todd Member
|
posted 06-08-2005 07:02 PM
Jack,Check with "Poly Esther" at DoDPI. She does ring in on the site from time to time although she has not posted in a while. I know LAPD prefers the standard "Since your application have you....?" I am pretty sure they are still using it. By the way, spell Esther's name correctly or......well you know! Ted IP: Logged |
Barry C Member
|
posted 06-08-2005 09:10 PM
Are you sure Ted? "Since your application ..." would apply to fairly recent conduct, which isn't much time to make a CQ do what it is supposed to do. In addition, it doesn't set the RQs apart, if that is the point. After all, does it matter if the guy raped a person before the application or after he filed it? You're interested in both time periods, so the "bar" doesn't bar the CQ from the RQs.IP: Logged |
Ted Todd Member
|
posted 06-09-2005 08:10 AM
Barry,I Wrote it wrong....It reads "Prior to you application". I do agree with you however that it still puts things way to close. Ted IP: Logged |
J L Ogilvie Moderator
|
posted 06-09-2005 12:20 PM
Ted, I think I know Esther's opinion (according to DODpi)but I hoped she or someone else would reply here.I don't know what you are referring to about Esther? She is the sweetest person and wouldn't harm anyone. Would you Esther? Jack ------------------
IP: Logged |
polyestr Member
|
posted 06-13-2005 07:03 AM
Hi all,I am sorry I have not posted lately, but I do check in on your postings at least once a week. First of all, you are right Jack I am SWEET. Now regarding the use of no time bars in employment screening exams. The Law Enforcement Preemployment Test (LEPET) as taught at DoDPI, teaches that some comparison questions do not require time bars because they don't have the potential to "bleed" over into the relevant issues. Examples of questions not needing time bars are: 1. Did you ever take credit for someone else’s work? 2. Did you ever lie about someone behind his or her back? 3. Did you ever cheat in school? 4. Did you ever cheat in sports? 5. Did you ever cheat at cards? As I have already said, you can use comparisons with no time bars as long as there is NO potential that the questions can become relevant. For example, one of your questions is "Are you intentionally falsifying or omitting any information on your application forms?” Based on that question, I would not ask a question like "Have you ever lied on an official document?" and not use a time bar because if they lied on the application, that commparison now could become relevant. Hope that helps out. Wishing you all a great week. Esther IP: Logged |
Lieguy Member
|
posted 06-14-2005 02:12 PM
For what it's worth, I also use the comparisons that were introduced by Don Kraphol, "Are you the type of person who..." and I believe he called them hypothetical controls. They work particularly well in testing juveniles, as time bars are hard to set with kids anyway.------------------ A Half Truth is a Whole Lie IP: Logged |
Poly761 Member
|
posted 06-27-2005 01:42 PM
I have not "barred" pre-employment test questions, left each issue wide open for discussion during question construction. Isn't there too much risk information of concern could be excluded?..... IP: Logged |
Barry C Member
|
posted 06-27-2005 02:23 PM
I think you're talking about relevant questions - not comparison questions. But, no, I wouldn't expect you'd want to bar RQs unless some policy, statute, ordinance, or case law, etc., prohibits you from doing otherwise. (Some courts have said you can't ask about marijuana use, for example, that took place more than six months ago.)IP: Logged |
Poly761 Member
|
posted 06-27-2005 05:15 PM
Appears I need to "get up to speed" with everyone but I don't/haven't used comparison/control questions in a pre-employment test (PET). It's my understanding once an opinion is "DI" to a PET question (some) examiners administer follow-up tests w/CQ's. I agree with doing everything possible to verify a "DI" opinion. I have concerns about being able to structure an adequate control. EXAMPLE: Q. Have you committed any undetected crime? If the examinee responds "yes" during pre-test, identifies the crime(s), the question is then rephrased "Other than what we discussed - ." If there now is a "DI" to this question how do you propose structuring an adequate control for a PET. (If) additional tests are administered what is gained other than another likely "DI" opinion. In my experience verification is received by admission/confession. In the end, isn't the best result an examiner can report in the form of an admission/confession? Thanks.....
IP: Logged |
Capstun Member
|
posted 06-27-2005 05:53 PM
Our agency uses the federal LEPET format for pre-employment screening. It is basically a modified MGQT. When I came here, they were using a bastardized Army I/R with some quasi-controls. I think it was taught by Dick Arthur. I know some agencies in the Northwest use it too, but I found it unacceptable. As I understand it, the Army abandoned it in the early 1970's, but for some reason it caught on in law enforcement. IP: Logged |
Barry C Member
|
posted 06-28-2005 10:12 PM
Poly761,In a screening exam, nobody is DI as reactions during a screening exam might be the result of a number of things, especially in an R/I format. I run a lot of follow up tests, and most are cleared - not DI'd. As for comparisons, you've got to be creative. You can always use lies. After all, you're not going to get admissions to them all. There are a number of studies out thhere showing everybody lies from 2 to 25+ times per day, so you can find a comparison if you try hard enough. How about a nice set up about FBI profiles and studies, blah, blah, blah, and ask something like "During those crimes you said you committed, did you ever get a thrill from doing them?" or something along those lines. I've never tried this one, but I'm sure you could make something work. After all anybody who got a thrill from stealing a candy bar would probably do it again, right? Robbing a bank isn't a far cry from there either. Do you see a possibility here? Your examinee won't know your full of baloney even if he's read the anti sites. Remember, ASTM standards don't allow for a DI call based on a screening exam alone. You've got to follow it up. Admissions are nice, but what if there really isn't anything to admit? Good luck. IP: Logged |
Poly761 Member
|
posted 06-29-2005 03:43 PM
Barry C -Referring to my earlier (undetected crime) example. My concern in the proper construction of a control question relative to a PET follow-up is based on two factors. First. The examiner would not know the type of (undetected)crime the examinee may have been involved in. In my earlier example the examinee admits to an undetected crime during pre-test, the question is then phrased to eliminate the admitted information (OTWWD). If, after initial tests it appears the examinee is “DI” in their response denying involvement in any undetected crime, what subject matter would you use in the follow-up control(s). We would not know if it could be related to theft, assault, sex, etc? My point, since the examiner does not know what type of crime could be involved, how can the controls be properly worded? Is a different criteria used in establishing (these) controls? Second. At what point in time would the “time bar” be established? In my example, if the admitted crime occurred during 2001 & this test was administered during July 2005, where would the “time bar” be set? It’s my understanding “time bars” are to be established for the control (before) the date/time of the suspected crime to eliminate contamination. Since we would not know when the suspected crime occurred what is used to determine at what point in time the “time bar” is set for the control question? What is the “ASTM” document number you refer to pertaining to a “DI call?” I’ve been away from testing for a couple years and I’m only familiar with EPPA, APA , State regulations and my training pertaining to examination procedures. ..... IP: Logged |
Barry C Member
|
posted 06-29-2005 05:12 PM
I just wrote a nice response that somehow I deleted, and I don't have the time to re-write it now.You shouldn't be calling a person DI during a screening test for the reasons I stated above. Your CQs don't have to be similar in nature to the particular crime in question - you woun't know anyhow - and it might not exist. You can use guilt / shame, lies, etc. As a matter of fact, the DoDPI ZCT requires lie CQs in a child sex case, and they work. I use guilt / shame, lies, and hurt / harm in murder cases. One is close, the others aren't, but they don't have to be three questions similar to the issue at hand. More later... IP: Logged |
Barry C Member
|
posted 06-30-2005 07:54 AM
The ASTM standard is E 2386-04. It says you need to run a single-issue test - not a screening exam - to render a DI opinion. (It also limits RQs to five at most in a screening exam.)IP: Logged |
Poly761 Member
|
posted 06-30-2005 07:00 PM
Barry C -Why is a follow-up test conducted subsequent to a screening/pre-employment examination (with control questions) if not due to the fact the examinee appears “DI” in their initial tests? I believe we’re now back to “square one.” My concern is about our ability (given this example) to properly construct a control question as we don’t know the nature of the possible/suspected crime. I appreciate your feedback but it appears we have a difference of professional opinion. I’ve always been advised control questions must be appropriate to the incident upon which the examination is based and can be identified as a “probable lie.” I’m checking further with APA and ASTM personnel in an attempt to clear this issue. When you have the time I’d appreciate it if you would list out questions you would ask for the example I have cited. During pre-test an examinee admits to an undetected crime, the test question is then structured (OTWWD have you committed any undetected crime). This hopefully eliminates the admission yet they are “DI” to the modified test question on 3 consecutive charts. I’m most interested in learning the “control questions” that would be used. …..
IP: Logged |
Barry C Member
|
posted 06-30-2005 08:36 PM
Again, nobody should be called DI on a screening exam. There are too many factors that could contribute to such reactions. So to say a person "looks DI" on a screening exam is not appropriate, which is why ASTM did what they did. (I'm an ASTM member.) The APA is trying to figure out what they are going to do. As of now, they don't have any screening exam standards. They have been promoting Don Krapohl's "succesive hurdles" model in that they've had him as a speaker on the topic at a number of their seminars. That is the approach I (and ASTM take). It is based on a scientific approach to the problem.What you must realize is validity for a screening exam is very low compared to s specific-issue criminal test. Some studies show tecniques that are no better than chance. The R/I is a great test to ID the deceptive if you limit the relevant questions to four - or at most five - in any one test. (I prefer three.) BUT, it is lousy at identifying the truthful. (And, it has almost no support in the scientific community, but I still like it in many circumstances.) Let's assume a screening test is about 70% accurate, which is probably pretty close, but there's not much research to say either way. (Unless you believe the NAS study, which said it's worthless - a position I don't agree with.) (Keep in mind some of the government studies only had chance level results in IDing the truthful.) If that's the case, you're going to call 3 out of 10 wrong, and the data out there supports the position that most of those errors will be false positives. If we know that - and we do - then we have a professional responsibility to do all we can to be fair to the truthful while still IDing the (truly) DI. The way you do that is to create a screening test with high sensitivity (to deception). You do that at the expense of lowering the specificity. To remedy that problem, you run those who react to RQs through more confirmatory testing. Don uses the SARS example to demonstrate, but I'll use TB. When you learn that a regular customer on the street tested positive for TB the first thing you do is find out who has had ANY type contact with the person. That is, you screen out those who couldn't have been infected because they never came within a mile of the person. Of course, you won't treat each person for TB yet as you've probably ID'd a large group of people, most of whom won't have TB. So, next you do the quick TB test. Those who get bumps on their arms (from the TB screening test) are considered tentatively positive for TB, but they are not DIAGNOSED as positive because a number of things could cause an error on the TB screening test. Instead, they are subjected to more diagnostic testing. It is only after a person tests positive on a diagnostic (highly accurate) test is one ID'd as having TB and treated. Otherwise, you'd be treating 100 people instead of the one guy who was unlucky enough to get the disease. Pre-employment exams are run the same way by most examiners these days. Otherwise, you end up calling truthful people DI at an unacceptable rate. Now, to get to your other question. One of the most researched tests out there is the Utah test, and they don't use three similar CQs. The PCSOT people are divided: some use sex CQs, others use lies. In murder cases you don't ask "NCWTC have you ever killed anybody?" because it's not a probable lie, but it is the only thing similar in nature to the issue at hand. A "hurt" CQ really doesn't compare (objectively), but they work, as do lies, and a bunch of other things. Now for an example: 1 Is today Thursday? 2 Sac Rel 3 (CQ) Are you the type of person who would lie to benefit yourself in any way. Or, Not connected with this issue (undisclosed crimes), DYRE telling a serious lie? 4 (R) Have you com, any serious crimes you haven't told me about? 5 (CQ) NCWTI, DYRE engaging in any non-criminal behaviors that the community might consider unethical. (Crimes are unethical, right? I've never used this one, but I'll try it.) 6 (R) Are you withholding any information about your involvement in any serious crimes? 7 (CQ) Do you know of any reason why you are not the best candidate for this job... shouldn't be hired, etc. There's one off the top of my head. I can't say it's perfect, but I pick my CQs based on the person I've got in front of me. Remember: the theory behind polygraph is that the truthful find the CQs more salient; and the decptive, the RQs. It doesn't really matter what they are as long as you set it up so that a truthful person has a place to focus his concerns. If you set it up correctly, it'll work. Also remember that a CQ should be a probable lie OR a question to which the examinee is UNCERTAIN. Could you ever be certain of a question such as "Are you the type of person who is completely trustworthy?" If you're lying about an undisclosed crime, you wouldn't care. If you didn't do the RQ, then the CQs stand right out there. IP: Logged |
J.B. McCloughan Member
|
posted 07-01-2005 09:36 AM
Not sure if this format will work for you for a “break down” examination, as I do not conduct PET.1 Is this the month of January? 2 Regarding breaking any other laws not disclosed, do you intend to answer each question truthfully? 3 Are you completely convinced that I won’t ask you any surprise questions? C4 Did you ever lie to stay out of trouble not disclosed? R5 Did you break any other laws not disclosed? C6 Did you ever think about breaking a law not discussed? R7 Did you break any other laws not disclosed that would DQ you? C8 Did you ever violate the trust of someone close to you not disclosed? 9 Is there anything else that you are afraid I will ask you a question about, even though I have told you I would not? Other CQs: Did you ever falsify anything for personal gain? (set up with how some people falsify their taxes, time sheets, vehicle purchase price, internet purchases, etc. to get more money or avoid taxes) Did you ever cheat on anything to advance your status? Did you ever lie to someone of authority to help yourself or someone you know? Did you ever think to do something unethical for personal gain? Did you ever do anything that could hurt your reputation in your community? I believe you are both correct on some of your points. Barry wants to insure standards are complied with and recommends a follow-up general specific issue test to add a quasi-scientific construct to a diagnostic test. Poly761 is concerned with an admission during the pre-test and how to deal with it to insure nothing further is present but is equally concerned about the separation of the admission and constructing viable comparison questions. The severity of the admission and the length or depth of discussion on that area or areas can adversely effect the salience of related areas of questioning on the examinations. He also, from his posts, seems to well realize that the relevant issue is potential and unknown in screening, as he appears concerned about the ability to separate admissions from further inquiry into that area. A TB screening tests for a specific condition. With polygraph screening we do not know what the results have discovered out of the vast unknown. A closer comparison would be with that of the MMPI or other psychological screening batteries. Screening is diagnostic and, due to the large area of possible diagnosis, the confidence of diagnosis is general even with a “break down test”. It should hypothetically be easier to eliminate a diagnosed area than to specifically identify its origin. The end phase of the screening should include a stage of investigation into the specific area of concern to establish whether or not there is an issue in the subjects past related to the area. Some do not utilize this end phase due to time and cost. However, the polygraph would still save both time and resources when utilizing this method, as it narrows the scope of inquiry for the background investigation. All in all this is just my take on the issue and I am not inferring that it is necessarily the best. To my knowledge, at this time there are no prescribed standardized methods for polygraph screening that have been adequately tried and tested. Because of this, it would seem to me that it is the given agencies HR section’s job to insure that any method being utilized for the purpose of selection meets any legal limitations and is reliably defendable upon challenge. There are at this time no national standards that have legal and binding authority over polygraph. Some states have licensing requirements and some do not. Some states have an educational requirement of a bachelor degree in a specified field of study and some a high school diploma or equivalent will suffice. Some require continuing education and some do not. Federal regulation would be nice but I’m not sure all would agree with me on that issue.
[This message has been edited by J.B. McCloughan (edited 07-01-2005).] IP: Logged |
Barry C Member
|
posted 07-01-2005 12:58 PM
I agree, and I like your analogy better.I also would add that the general crime question is going to be one of the toughest. Some break down a screening exam into a format designed to show where the area of concern might be, and then do a single issue exam on what ever comes out of that. Again, it takes longer to do, and you are going to have to have discussions between tests because you are looking for admissions. The tough part is not overstimulating the examinee to the relevant issue in the event he is in fact truthful. I think the APA and AAPP need to come up with some standards, probably in line with ASTM's. Again though, standards are nice, but there's not a lot to base them on (research wise). We do know enough to know some of what we should be avoiding though. I'd be cautious about your R7. (I know it's just an example, and you don't run those tests.) The DQ part would probably get anybody thinking, and therefore, reacting. I've used the question "Do you know of any reason why you should not get this job?" as a CQ, and it works. Candidates lie about dumb things because they think everything will DQ them. With all that, Poly761, do you think you could run a follow-up test now? IP: Logged |
Poly761 Member
|
posted 07-01-2005 11:42 PM
Barry C J.B. McCloughanThanks to both of you for your feedback. Yes, I believe I'd be more comfortable now in conducting a PE follow-up using CQ's. I agree a standard (format) needs to be established. While the two different PE follow-up formats I received were very helpful, they were different. I also reviewed a copy of Baxter's PE Re-Test that appears based on his "You" phase test, yet another format. Thanks again -
IP: Logged |
Barry C Member
|
posted 07-02-2005 08:13 AM
FYI, Don Krapohl wrote an article in his Best Practices series on these types of tests, rules, formats, etc., which was carried in APA, AAPP and CAPP journals. I think it appeared last year. You might find it very helpful. (Essentially he suggests picking a format you know, i.e., a recognized technique taught by a recognized school, and working around it as necessary in these types of tests.) If you don't have it, email him and he'll get it to you ASAP. I don't have his address off the top of my head, but it's probably on the DoDPI website or somebody here will know it. (I've got it saved at work - not here.) IP: Logged |
Ted Todd Member
|
posted 07-02-2005 09:11 AM
Don can be found at DoDPI. I also beleive his company is called National Polygraph Consultants.Ted [This message has been edited by Ted Todd (edited 07-02-2005).] IP: Logged | |